Literature Search Made Easy

M any of us have been faced with this situation: we found a certain field of research intriguing, went to an online archive to search for related papers, and the results were just less than satisfactory: they are too unstructured to obtain a big picture, such as how this field evolves over time, what sub-fields it is composed of. Instead of relying on human instinct for such insights, our project provides an alternative: using machine learning algorithms to peruse thru online archives, generate the information of interest, and present them in a human friendly way. In this project, we used hierarchical latent Dirichlet allocation (hLDA) to find the topics and their hierarchy and dynamic topic model (DTM) to find the time evolution of topics. In addition to using these two models, we also made an important extension to them constructing the corpus with key phrases instead of keywords. Furthermore, we devised an algorithm to remove the duplication in word and phrase count, which enabled us to generate a corpus composed of both keywords and key phrases. This approach allowed us to greatly improve the interpretability of the result since a large number of concepts in science and technology are formulated as phrases. In our experiments with 14185 condensed matter physics papers from arXiv, the above methods effectively found the trend and hierarchy of research topics and revealed some very interesting insights. In our report, the Trends in Research Topics section demonstrates the analysis of topic trends, advantage of using key phrases versus keywords and a caveat in word and phrase count when constructing the corpus. The Topic Hierarchy section presents the generation of a hierarchy of topics and the solution to the caveat aforementioned.